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DETAILED ACTION 
Drawings 

1 . New corrected drawings in compliance with 37 CFR 1 .121 (d) are required in this 
application because of Draftperson's Review. Applicant is advised to employ the 
services of a competent patent draftsperson outside the Office, as the U.S. Patent and 
Trademark Office no longer prepares new drawings. The corrected drawings are 
required in reply to the Office action to avoid abandonment of the application. The 
requirement for corrected drawings will not be held in abeyance. 

Claim Objections 

2. Claim 1 is objected to because of the following informalities: Line 5, states "are 
are". Appropriate correction is required. 

3. Claim 1 1 and 31 are objected to because of the following informalities: Line 2, 
states "voice command input including.". Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 
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5. Claims 1-6, 22-26. 41 , 42. 44 and 45 are rejected under 35 U.S.C. 102(b) as 
being anticipated by "Multimodal User Interfaces in the Open Agent Architecture" by 
MORANetal.. 

As to claim 1 MORAN teaches a collaborative community of distributed electronic 
agents (agents), organized to provide a mobile computing environment (pg. 1, "Only the 
primary user interface agents need run on the local computer, thereby simplifying the 
task of using a range of applications from a variety of platforms, especially low-powered 
computers such as Personal Digital Assistants (PDAs)."), the community of distributed 
electronic agents (agents) comprising: an agent registry (memory / storage used by 
facilitator for storing agent capabilities) wherein one or more capabilities of each of the 
electronic agents are registered in the form of an inter-agent communication language 
(ICL) (via registering with the facilitator) (pg. 4, "When an agent is added to the 
application, it registers its capabilities with the Facilitator... When an agent needs work 
done by other agents within the application, it sends a request to the Facilitator which 
then delegates it to an agent or agents, that have registered that they can handle the 
needed tasks."); a facilitator agent (Facilitator) arranged to coordinate cooperative task 
completion among the electronic agents (agents) by delegating one or more received 
ICL goals (requests) to a selected one or more electronic agents (agents) based upon 
the registered capabilities of the selected agents (pg. 4, "When an agent is added to the 
application, it registers its capabilities with the Facilitator... When an agent needs work 
done by other agents within the application, it sends a request to the Facilitator which 
then delegates it to an agent or agents, that have registered that they can handle the 
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needed tasks."); one or more service-providing electronic agents (agents), being in bi- 
directional communication with the facilitator agent (Facilitator), including at least one 
location agent (calendar agent) operable to ascertain a current physical location of a 
user (pg. 5, "The notify action involves... For example, if the user is in a meeting in a 
conference room, the Notify agent first determines his current location by checking his 
calendar..."); and one or more computer interface agents (User Interface Agents) being 
in bi-directional communication with the facilitator agent (Facilitator), the mobile 
computer interface agents (User Interface Agents) being operable to process at least 
one mobile user input type (modality / speech / pen-based input) and to responsively 
generate and present to the facilitator agent (Facilitator) one or more ICL goals 
(request) corresponding to the user's desired request, whereby the mobile user is 
capable of accessing both local and remote resources (appropriate application agents) 
(pg. 2, "The user interface is implemented with a set of agents that have at their logical 
center an agent called the User Interface (Ul) Agent. The User Interface Agent 
manages the various modalities and applies additional interpretation to those inputs as 
needed... When the Ul Agent detects speech or pen-based input, it invokes a speech 
recognition agent or handwriting recognition agent, and sends the text returned by that 
agent to a natural language understanding agent, which produces a logical form 
representation of the user's request. This logical form is then passed to a 
Facilitator... which is then passed to the Facilitator agent for delegation to the 
appropriate application agents."). 



Application/Control Number: 09/475,092 Page 5 

Art Unit: 2195 

As to claim 41 , MORAN teaches a method for proactively providing a mobile user 
with location-sensitive navigational infonnation, the method comprising the acts of: 
setting an automatic software trigger (trigger), the trigger being condition upon detecting 
a specified condition (pg. 5, "The OAA supports triggers in which both the condition and 
action parts of a request can cover the full range of functionality represented by the 
agents dynamically connected to the network."); detecting the specified condition (pg. 5, 
"When an advertisement meeting the specified criteria is detected, a request is sent to 
the Facilitator for a notify action to be delegated to the appropriate other agents."); using 
an electronic location agent (Calendar agent) to ascertain the mobile user's current 
location (pg. 5, "The notify action involves... For example, if the user is in a meeting in a 
conference room, the Notify agent first determines his current location by checking his 
calendar..."); and in response to the software trigger (trigger), using one or more 
interface agents (User Interface Agents) to present navigational information to the user 
relative to the user's current location (pg. 6, "Requests expressed in a variety of 
modalities can control the scrolling and zoom level of the map, retrieve information 
about locations and distances, display hotels or attractions meeting a user's 
preferences, or present detailed information in a variety of media about particular hotels 
or attractions."). 

As to claim 45, MORAN teaches a highly mobile, ambient computing 
environment utilizing a community of distributed electronic agents (agents), the 
computer environment (pg. 1, "Only the primary user interface agents need run on the 
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local computer, thereby simplifying the task of using a range of applications from a 
variety of platforms, especially low-powered computers such as Personal Digital 
Assistants (PDAs).") comprising: a plurality of autonomous sen/ice-providing electronic 
agents (agents) associated with available resources (data / capabilities / maps); a 
facilitator agent (Facilitator) arranged to coordinate cooperative task completion utilizing 
the plurality of autonomous service-providing electronic agents (pg. 4, *When an agent 
is added to the application, it registers its capabilities with the Facilitator... When an 
agent needs work done by other agents within the application, it sends a request to the 
Facilitator which then delegates it to an agent or agents, that have registered that they 
can handle the needed tasks."); and a mobile computer interface (User Interface Agent) 
responsive to a plurality of user input types (modalities / speech / pen-based input), the 
mobile computer interface being in bi-directional communication with the facilitator 
agent (Facilitator), the mobile computer interface operable to forward a user request for 
resource access to the facilitator agent for processing, the mobile computer interface 
further operable to provide the user the requested resource access as provided by the 
facilitator agent, whereby the mobile user is capable of accessing both local and remote 
resources (pg. 2, "The user interface is implemented with a set of agents that have at 
their logical center an agent called the User Interface (Ul) Agent. The User Interface 
Agent manages the various modalities and applies additional interpretation to those 
inputs as needed... When the Ul Agent detects speech or pen-based input, it invokes a 
speech recognition agent or handwriting recognition agent, and sends the text retumed 
by that agent to a natural language understanding agent, which produces a logical form 
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representation of the user's request. This logical fornn is then passed to a 
Facilitator... which is then passed to the Facilitator agent for delegation to the 
appropriate application agents."). 

As to claims 2-4, MORAN teaches the mobile computer interface agents (User 
Interface Agents) and location agent (location agent / agent handling maps) are 
collaboratively operable to provide Multi-modal Map information (Map-based Tourist 
Information) relative to the user's current location wherein the map information includes 
spoken directions (voice commands) by means of a text-to-speech output agent 
(Speech Recognition Agent) (pg. 2, 'When the Ul agent detects speech or pen-based 
input, it invokes a speech recognition agent or handwriting recognition agent, and sends 
the text returned by that agent to a natural language understanding agent, which 
produces a logical form representation of the user's request. This logical form is then 
passed to a Facilitator agent, which identifies the subtasks and delegates them to the 
appropriate agents. For example, in our Map-based Tourist Information.... The locations 
of the two places are managed by different agents, and the distance calculation is 
performed by yet another agent"; see also pg. 6, Map-based Tourist Information) 

As to claim 5-6, MORAN teaches one or more conditional triggers (triggers) 
associated with one or more of the electronic service agents (agents), and wherein the 
community is collaboratively operable to interact with the user based in part by the 
location agent (calendar agent) in response to the triggers condition (pg. 5, "The OAA 
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supports triggers in which both the cx)ndition and action parts of a request can cover the 
full range of functionality represented by the agents dynamically connected to the 
network/'; pg. 5, "The notify action involves... For example, if the user is in a meeting in 
a conference room, the Notify agent first determines his current location by checking his 
calendar..."). 

As to claim 20, MORAN teaches the at least one mobile user input type is 
selected from the following group: speech, gestures, pen/stylus input, and touch-screen 
input (pg. 2, "When the Ul Agent detects speech or pen-based input..."). 

As to claim 21, MORAN teaches the mobile computing environment is provided 
in a handheld computing device (PDA / lightweight portable computer) (pg. 1, "Mobile 
section). 

As to claim 22-26, reference is made to a method that corresponds to the system 
of claims 1-4 and 6 and therefore is met by the rejection of claims 1-4 and 6 as detailed 
above. However, claim 22 further discloses that the performance by the selected 
electronic agents, generate one or more new ICL goals for processing by the facilitator 
agent and processing all ICL sub-goals until the original goal is satisfied, MORAN 
teaches that the performance of requests by selected electronic agents (agents), the 
generation of one or more new ICL goals for processing by the facilitator agent 
(Facilitator) and processing all ICL sub-goals (via distributing to the appropriate agent) 
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(pg. 2, "The user interface is implemented with a set of agents that have at their logical 
center an agent called the User Interface (Ul) Agent. The User Interface Agent 
manages the various modalities and applies additional interpretation to those inputs as 
needed... When the Ul Agent detects speech or pen-based input, it invokes a speech 
recognition agent or handwriting recognition agent, and sends the text returned by that 
agent to a natural language understanding agent, which produces a logical form 
representation of the user's request. This logical form is then passed to a 
Facilitator... which is then passed to the Facilitator agent for delegation to the 
appropriate application agents."). It would be inherent within the teachings of MORAN 
that sub-request of the overall request are handled in the same manner as disclosed 
with the original request since the Facilitator functionality is to decompose complex 
requests and translate the terminology used (pg. 4) and a sub-request is a parent 
request in its sub-processing. 

As to claim 42, MORAN teaches the navigational information includes one or 
more of the following: visual display of a map (pg. 6, Map-based Tourist Information 
section) and spoken directions (pg. 6, "Map-based interfaces provide a rich setting in 
which to explore the coordination of gesture with speech and traditional GUI 
modalities.... Display of hotels can be obtained by writing or speaking."). 

As to claim 44, MORAN teaches the specified trigger condition includes the 
contents of an electronic calendar (pg. 5, "The notify action involves... For example, if 
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the user is in a meeting in a conference room, the Notify agent first determines his 
current location by checking his calendar..."). 



Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 7-19, 27-39 and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Multimodal User Interfaces in the Open Agent Architecture" by 
MORAN et al. 

As to claims 7-19, MORAN teaches user interfaces agents execute on mobile 
computing environments and are used to control a range of applications from a variety 
of platforms (abstract), by receiving voice input or pen-based inputs, converting the 
input to a logical form representation, that is then passed to a Facilitator agent which 
identifies the subtasks based on the registered capabilities and delegates them to the 
appropriate agents (pg. 2, The User Interface). It is inherent within the teachings of 
MORAN that pen-based inputs require a visual display that is manipulated by the user. 
However, MORAN does not teach that the various services control electronic devices, 
an email system, a telephone system, a voicemail system, a game system, access to 
the Internet, a television system, car manuals, or diagnostic infomnation. Official Notice 
is taken in that the various systems are well known in the art and therefore would be 
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obvious in view of MORAN in order to provide agent interaction and execution of the 
requested service. For instance, it is well known in the art that a cell phone, a type of 
mobile device, has an interface for games, the Internet, voicemail, and dialing functions. 
A remote control device has functions for interfacing with a television. In addition, 
online car manuals and diagnostic information systems wherein a user enters a 
password to access this information is also well known in the art, as well as, computers 
implemented into the dash of an automobile for global positioning and displaying related 
information about sensors of the automobile. 

As to claim 27-39, reference is made to a method that corresponds to the system 
of claims 7-19 and therefore is met by the rejection of claims 7-19 as detailed above. 

As to claim 43, MORAN substantially discloses the invention above. However. 
MORAN does not explicitly teach the status of a vehicle sensor. Official Notice is taken 
in that it is well known in the art at the time of the invention that vehicles have global 
positioning sensors within vehicles that detail the location of a user, i.e. the driver. It 
would be obvious in view of MORAN that the when the user asks for the distance from 
here to here, that the vehicle global positioning is consulted to answer the request. 
Therefore, it would be obvious to one skilled in the art to combine MORAN with the well 
known global positioning sensors of vehicles to handle a mobile request. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Lewis A. Bullock, Jr. whose telephone number is (571) 
272-3759. The examiner can normally be reached on Monday-Friday, 8:30 - 5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Meng An can be reached on (571) 272-3756. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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